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Abstract 

We study the quantization problem for certain types of jump processes. The prob- 
abihties for the number of jumps are assumed to be bounded by Poisson weights. 
Otherwise, jump positions and increments can be rather generally distributed and 
correlated. We show in particular that in many cases entropy coding error and quan- 
tization error have distinct rates. Finally, we investigate the quantization problem 
for the special case of M*^- valued compound Poisson processes. 
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1 Introduction and results 

1.1 Statement of the problem 

In this article, we study the quantization- and entropy coding problem for certain types 
of jump processes. Given a random variable X, the aim is to find a good approximation 
X to X that satisfies a particular complexity constraint. 

Let s > 0, X be a random variable in a measurable space {E,S), p a distortion measure 
on E (i.e. a measurable, symmetric function p : E x E —>■ M>o with p{x, y) = iS x = y), 
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and r > 0. Then we define the quantization error as follows: 

l/s 



D^'i\r I X,p,s) := inf |(^Eniinp(X,a)'y 



: log #C < r ^ . 



The number D'^'?^ represents the best-achievable average error when encoding the signal 
X with r nats. The term 'nats' is used instead of 'bits', since we calculate the amount of 
information using the natural logarithm. Further, we investigate the entropy coding error, 
which can be understood as the average error when encoding the signal X using - on 
average - r nats: 

D^''\r I X,p, s) := inf | (Ep(X, X)")^^' : X random var. with H{X) < r| , 
where H is the (discrete) entropy of a random variable: 



H{X) :- 



• Ex P = a;) log F{X = x) X discrete, 
oo otherwise. 



In slight misuse of notation we also write D^'^\r \ X, \\.\\ ,s) if p{x,y) = \\x — y\\ for a 
norm distortion ||.||. Analogously, we deal with the entropy coding error. We recall that 
L)(^)(r I X,p,s) (r I X,p,s). 

The problems described above arise naturally in coding theory, where for instance, the 
complexity of a signal has to be reduced due to capacity restrictions of a channel or 
simply (lossy) data compression is considered (see for instance [6] for a general account 
on coding theory and [15] for a historic outline of the information constraints). Beyond 
these information-theoretic applications, the quantization error is tightly related to certain 
quadrature problems: the quantization error can be defined equivalently as the worst-case 
error of a particular quadrature problem. Moreover, further quadrature problems are 
linked to the quantization problem via estimates involving both quantities. Recent results 
in that direction can be found in [7] (see also [19] for earlier results). 

The analysis of the quantization- and entropy coding error started in the 40s of the 20th 
century. At that time research was mainly focused on finite-dimensional signals; and the 
numerous publications mainly appeared in the engineering literature. A mathematical 
account of the results for finite-dimensional signals is provided by [13]. Since about 2000 
researchers are attracted by the problem in the case where the original signal is infinite- 
dimensional. A series of articles followed on (infinite-dimensional) random vectors X that 
are Gaussian (see for instance [11], [16], [12]), diffusions ([H], [9], [ID]), and Levy processes 



In this article, we provide asymptotic estimates for the quantization- and entropy coding 
error for certain jump processes. The results are shown to be sharp in several cases. 
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In contrast to the (infinite-dimensional) settings studied before, there is a quahtative 
difference in the (best-achievable) approximation error induced by the two constraints. 



1.2 Some notation and the model 

Let us now introduce the jump processes that we investigate in this article. 

We define the space D([0, ![,£') to be the space of all functions / : [0,1[-^ E that are 
piecewise constant and possess a finite number of jumps, where if / has a jump at t from 
the value a e E to b e E, then f{t) — b. We endow D([0, 1[, E) with the cr-field induced 
by the projections. 

In the sequel, X = (X(t))tg[o,i[ denotes a D([0, 1[, £')-valued random vector. We denote 
by Nx the random number of jumps of X, let < Yi < • • • < Y^^ < be the jump 
positions of X, and set Yq = and l^x+i — 1- Moreover, we denote by 

Zr.= p{X{Y,_,),X{Yi)) 

the moduli of the increments and, in the case where is a linear space, we denote by 

Z« ■.= X{Y^-X{Y,_,) 

the increments. 

As distortion measure on D([0, 1[, E) we consider 

Pu{f,g):= rp{f{t),g{t))dt, f,ge-D{[0,l[,E), (1) 
Jo 

where p is a distortion measure on E. It is straightforward to extend the results of this 
paper to the distortion measure (^{f,g) — {Jq p{f (t) , g{t)y dty^^ , with 1 < p < oo. 

Our lower bounds require that the jump positions constitute a Poisson point process with 
intensity A. The upper bounds on the complexity are proven under weaker assumptions 
on X. Here, we only assume that the total number of jumps can be estimated against the 
probability weights of a Poisson random variable: 

^{Nx = k)<—e-^K, k>0, (2) 

where A > and K > 1 are some fixed parameters. In particular, one can choose K — 1, 
if the jump positions are induced by a Poisson point process. 

Sometimes we shall also impose the following condition: 
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(*) The jump positions are independent of the jump destinations, which means that, 
given the event {Nx = k}, the vector (Yi,...,Yfc) is independent of the vector 

(x(Fo),...,x(n)). 

Let us introduce some more notation. Firstly, we make use of the concept of metric entropy. 
If p is a distortion measure on E we define its covering numbers by 

N{E, p, e) := min{n G N : 3xi, . . . , Xn & E \/x e E 3i : p{x, Xi) < e}. 

A set {xi, . . . ,Xn} for which the defining property of holds is called an e-net of E. Note 
that in general one has to assume that N{E, p, e) is well-defined, i.e. that for all e > 
there is an e-net of E. This is ensured if, for example, {E, p) is a precompact metric space. 
We also introduce the inverse concept of which we call given by 

rf('')(e|X,p,s) := inf {n > l,n G N : ^('^^(logn | X,p,s) < e] . 

In other words, S'^^ is the number of points needed to quantize with error at most e, i.e. 
roughly it is the inverse function of D^'^'> {\og{.)) . 

We shall also need the notation of strong and weak asymptotics. Namely, we write f ^ g, 
if limsup f /g < 1. Analogously, f ^ g is defined. Furthermore, f ^ g means Ymvf/g = 1. 
We also use / ~ (? if < liminf f/g < limsup f/g < oo. Finally, throughout the article 
Xd denotes the rf-dimensional Lebesgue measure. 

The paper is organized as follows. In the rest of this section we state the main results. 
In Section II. 3[ we state the upper bounds for both quantities under various additional 
assumptions. In Section II. 4[ the upper bounds are complemented by corresponding lower 
bounds. In particular, we obtain that the upper and lower bounds are tight in many 
cases. Finally, Section 11.51 is devoted to the particular setting where X is a compound 
Poisson process. The proofs for the upper bounds can be found in Section [2J There, 
explicit coding strategies are constructed. The lower bounds are proven in Sections [3] and 
m for quantization- and entropy coding, respectively. The proofs for the lower bounds of 
the quantization error rely on a small ball argument, whereas the lower bounds for the 
entropy coding error are derived using the Shannon lower bound for a related problem. 



1.3 Upper bounds 

Our first result concerns the case where the space E has finite covering numbers. In 
the case where {E, p) is a metric space, this corresponds to the assumption that E is 
precompact. 
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Theorem 1 Assume that w := sup^ p{x,y) < oo and that the upper box dimension 




(3) 



is finite. Then 



logD(^)(r I X,pD,s 



)> 



2 



r logr. 



(4) 



s(l + 7) 



Theorem 2 ylsswme that N{E,p,e) < oo /or all e > and that w := snp^ p{x,y) < 
oo. 



(^aj For all r > Tq = ro(A) 



D(^) (ir (Ar + (A + 1) log A^(E, p, e 



")) I X,pD,s) <C,(u; + l)if^/^e-^ 



(5) 



where the constant Cg depends on s only, and K and A are the constants from 
(b ) In particular, if the jump positions are distributed according to a Poisson point process 
with rate \, we obtain for r > tq = ro(A) 



(c) In the case of a discrete space E = {xi, . . . ,Xq} we even have the more precise 



D'^^\K{Xr + {X + l)\ogq) \ X, p-o, s) < AK^^'w mm{l, Xy/' e-'' , (7) 
for r > Tq = ro(A). 



Theorem [2] can be interpreted in the foUowing way. In order to quantize with error one 
needs, on average, Ar nats to encode the jump positions, A log A^(i?, p, e"'') nats in order 
to encode the increments, and another log A^(i?, p, e~^) nats in order to encode the initial 
position X{fS). In particular, the same result can be proved without the log A^(i?, p, e"*^) 
term if the initial value of the process is deterministic. 

Let us compare Theorem [T] and Theorem [2] in the case where N{E,p,6) < qe~'^ . We point 
out that the asserted rate of the quantization error is different to the one of the entropy 
coding error. As we will see below neither the quantization error bounds nor the entropy 
coding error bounds can be improved significantly. 

Finally note that N{E,p,6) < oo for all £ > does not necessarily imply that w : = 
sup^ y^^ p{x, ?/) < oo if p does not satisfy the triangle inequality. 

For the remainder of this subsection, let us assume that {E, ||.||) is a normed linear space 
with distortion measure p{x,y) = \\x — y\\. We assume that the jump destinations of X 



D^'^ (Ar + (A + 1) log N{E, p, e"" 



) I X,pD,s) <C,(w7 + l)e-^ 



(6) 



estimate 
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(and thus the increments Z^^^) are independent of the jump positions (condition (*)). 
Furthermore, assume that the increments, conditioned upon Nx = k, are identically 
distributed (not necessarily independent among each other) with the same law as the 
i?- valued random variable, say, Z^^\ Furthermore we assume that X(0) is deterministic, 
i.e. that for some xq E E X(0) = xq a.s. 

Theorem 3 Under the above assumtions the following statements are true. 

(a) If 

7 := hmsup — G 0, oo , 

£-.0 log 1/e 

then is valid for the newly defined 7. 

(b) IfS''\e\Z^^\\\.\\,s) <oo for all e>0, then 

D(^)(K(Ar + Alogrf('?)(e-'^|Z«,||.||,s)) | X,pd,s) < C e-\ 

holds with some constant C > depending on the parameters s, K, X, and E| 1 

Theorem [3] relates the complexity of coding X to that of coding the increments. If the 
assumptions of both Theorems [2] and [3] are satisfied, then the bounds of the latter theorem 
provide a better estimate since in general d'-'i\e) < N^e) (see Lemma [20l) . However, 
note that in contrast to Theorems [1] and [21 Theorem [3] requires that the increments are 
identically distributed and independent of the jump positions. In case of Theorems [H and Ej 
this is not necessary since, by assumption, the space E is sufficiently well-structured (in 
the sense of small metric entropy A^). 

Let us remark that the assumption in Theorem [3] that X{0) be deterministic is for sim- 
plicity only. If instead X(0) is a random variable in E, one has to add d^'^\e \ X{0), p, s) 
to the average number of nats needed to encode X conditioned upon X{0). 

Finally we mention that one can also prove counterparts to assertions (b) and (c) of 
Theorem [2] in the setting of Theorem [31 



1.4 Lower bounds 



As an illustration consider the case of a discrete space E, namely let E = {xi, . . . ,Xq}, 
which was first studied in ^20j. Then N{E, p, e) < q, and we thus obtain from Theorems [1] 
and [2 



logD^'^\r\X,pn,s)>J-rlogr and - log D^'\Kr \ X, p-^, s) > j. (8) 
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Now we ask for lower bounds. Clearly, one cannot expect a non-trivial lower bound when 
only assuming ([2]). Thus, let us assume in this subsection that the jump positions con- 
stitute a Poisson point process and that condition (*) holds. In this case, we show in 
Theorem m that the order of D^'^^ in ([8]) is in fact the true order on this scale. Below, in 
Theorem [71 we show that the order of D^'^'^ is the correct one, too. 

We consider a more general situation than a finite, discrete space. We only have to assume 
that there is sufficient uncertainty in the model in order to ensure that every jump indeed 
has to be encoded. 

Concretely, assume that condition (*) holds and that the jump positions form a Poisson 
point process. Furthermore, we assume that, given the event that k jumps occur {{Nx = 
k}), the moduli of the increments Zi, . . . , Zk are such that there are Eq > and 6o > 
(independent of k) such that for alH = 1, . . . , fc, P {Zi > Eq \ Nx = k) > 5q. Additionally, 
we now impose that {E, p) is a metric space. 

Theorem 4 Under the above assumptions, 



-logDW)(r I X,pd,s) < y-rlogr. 
In particular, for a discrete metric space E = {xi, . . . , Xg}, 




Note that in view of ([7]) the rates for quantization error and entropy coding error must 
be different in case of a discrete metric space E = {xi, . . . ,Xq}. Moreover, the order of 
convergence of the quantization error depends strongly on the moment s. In particular, 
one has for two distinct moments < s < s' that 

,.^g'.'(r|X.po..) 

r^oo ]j{q){r I X,Pd, s') 

This contrasts earlier results on quantization where the same order of convergence is 
obtained for all moments s > 0. 

Let us consider a simple example. 

Example 5 Let X he an alternating Poisson process, i. e. 

N(t) 

x{t) = Y.i-n-\ (9) 

i=l 
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where (iV(i))tG[o,i] is a Poisson (counting) process with rate A (cf. Section [773]] with the 
natural metric \.\ on = {0, 1}. Then Theorem^ yields 



- log Z}W(r I X,pD,s) 




Remark 6 Recall that the assertions of the upper hounds, do not require that {E, p) is 
a metric space. The statement is valid for any distortion measure p. However, the lower 
bound from Theorem fails for general distortion measures. This can be seen from the 
following simple example. Let E = {0,1} U {2~",?2 > 1} and p(0, 1) = 1, p(0,2~") = 
p{l, 2~") = 2'^, p(2'^, 2~''") = 1 for n ^ m and n,m > 1. Note that this is not a metric 
space. 

Consider the alternating Poisson process, i.e. the model from Then X satisfies the 
assumptions of Theorem in particular those for the moduli of the increments (since 
Zi = l), but D^i\r I X,pd,s) = 0, for all r > 0. 

Next, we will prove a lower bound for the entropy coding error. 

Theorem 7 Let X be a jump process satisfying condition (*). We assume that he jump 
positions {Yi) form a Poisson point process with rate A. Moreover, we suppose that p 
defines a metric on E and that a.s. the moduli of the jumps are bounded from below by 
£0 > 0. Then for all s > 1 and all sufficiently large r 

D'^^\r I X,pD,s) >eoCmin(l,A)e~"/\ 

where C > is an absolute constant. 

Remark 8 The lower bound in Theorem is actually shown to hold for the distortion 
rate function D{r \ X,p,s) defined in Section\^ 

We obtain the following corollary ClS db special case. 

Corollary 9 Let X satisfy the conditions of Theorem^ Assume additionally that X{0) 
is deterministic and consider the case of a discrete metric space E = {xi, . . . ,Xg} with 
w := maXx^y^E p{x,y) . Then for s > 1 

Ci£omin(l,A)e~"/^ < D^^^r \ X,pj^,s) < Cagw mm{l, X^^' e~'-/\ 

for large enough r and absolute constants Ci,C2 > 0. 

The corollary follows immediately from part (c) of Theorem [2] and the remark after it and 
Theorem [71 This result shows that the bounds for the entropy coding error in Theorems [2] 
and [3] are tight. 
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Example 10 Consider again the alternating Poisson process from ^ with the natural 
metric \.\. Then Corollary\^ yields, for all s > 1, 

Ci min(l, A)e-"/^ < D^'\r \ X,pd,s) < C2mm{l, X^/' e"'/\ 

for large enough r and absolute constants Ci, C2 > 0. 

Example 11 Let us illustrate the influence of a random initial position on our estimates. 
For this purpose, consider an alternating Poisson process with random initial position, i. e. 

N{t) 

x{t) =x(o) + 

i=l 

where X{0) equals and 1 with probability 1/2, respectively, cf. fWf . Our TheoremlE, part 
(c), and Theorem\^ yield 

Ci mm(l, A)e-"/^ < D^'\r \ X,pd,s) < C2 mm(l, A)^/^2i/^ e-''/\ 

for all s > 1 and all large enough r and absolute constants Ci, C2 > 0. 

1.5 Application to compound Poisson processes in M"^ 

As an application of our results, let us determine the coding complexity of M'^-valued 
compound Poisson processes. Recall that a Levy process with finite Levy measure is a 
compound Poisson process with the following structure, cf. e.g. [3]. 

Let {N{t))t>Q be a Poisson (counting) process with intensity A > 0, i.e. let N{t) : = 
max{n > : Z^ILi — where (cj) are i.i.d. standard exponential random variables. 
Consider 

N{t) 

x{t) = Y.z^'\ ie[o,i[, (10) 

i=l 

where the i = 1,2,..., are i.i.d. and distributed according to any probability dis- 
tribution in M'^ with P = 0^ =0. Note that this notation is consistent with the 
one employed above for the increments. Note furthermore that for compound Poisson 
processes condition (*) is satisfied. 



We consider the distortion measure 



l^lli:= ri|X(t)|Ldt, 





which of course coincides with pD for p = \\.\\^, where as usual := maxj=i_...^rf \xi\. 

However, one can replace W-W^^ by any norm on R'', which would change only the constants. 
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Theorem [3] yields the following corollary. 

Corollary 12 Let X be a compound Poisson process as defined in l[TO\) and s > 0. 
(a) Assume that 

y log^(^)(e|ZW,||.|L,.) 

7:=limsup ; — (11) 

e^o log l/e 

is finite. Then 

- log D^'^\r I X,pd,s) > ^j^^^^yrlogr. 

(h) Let d^'^\e\ Z'^^\ s) < oo for all e > 0. Then, for r > tq and a constant C = 
C(s,A,E||ZW||^), we have 

D^'\\r + {\ + l)d^''\e-'^\Z^'\p,s) \ X,pd,s) < Ce'^ 

Alternatively, one can study the consequences of Theorems [1] and [2] if one has additional 
information on the range of X. 

As for lower bounds we can apply Theorem HJ which gives the following. 

Corollary 13 Let X be a compound Poisson process as defined in fT^) and s > 0. Then 



-\ogD^''\r I X,Pd,s) < y-rlogr. 
// additionally l[Tl\) holds with 7 = 0, then 



log D^'^\r I X,p-£,,s) ~ \ -rlogr. 



We obtain a similar result in the case that the distribution of the increments has an 
absolutely continuous component. 

Theorem 14 Let X be a compound Poisson process as defined in [T^) and s > 0. Assume 
that the distribution of Z^^^ has an absolutely continuous component. Then 



\ogD^''\r I X,pd,s) < W—-^-— rlogr. 



s{l + d) 



If additionally ( fTTj) holds with ■j = d then 



AogD^'i^r I X,pr>,s) ~ y ^^^^^^ rlogr. 
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Theorems H] and [T3] show that the upper bound for the quantization rate in Theorems [T] 
and [3] (and thus Corollary [T2l) cannot be improved in general (for all 7 G N). 

Let us finally list a corollary of Theorem [71 

Corollary 15 Let X be a compound Poisson process as defined in ( fJOj) . Assume that 

Z^^^ > Eq a.s. Then, for all s > 1, 
00 

D^^\r I X,pD,s) >eoCmin(l,A)e-"/\ 
for r > Vq and C > an absolute constant. 

The most instructive examples of the application of the results of this subsection are given 
now. 

Example 16 Consider a Poisson (counting) process with intensity \, i.e. let Z^^^ = 1. 
Then 



- log D^''\r I X,p-D,s) ~ y^rlogr 

and 



Cimin(l,A)e-"/^ < D'^^\r \ X,pd,s) < C2e-''/\ 
for s > 1, r > ro, where Ci > is an absolute constant, and C2 > depends on s and X. 

Example 17 Let Z^^^ be uniformly distributed in [0, 1]*^. Then 



-logL)('?)(r I X,p-D,s) ~ y ^^Qg^ 

and 

for s > 1, r > Tq, where C2 > depends on s and A. We conjecture that the order on the 
right-hand side is the correct one. 

Example 18 Let Z^^^ be uniformly distributed in C , where C is the Cantor set in [0, 1]. 
S'ei 7 := log2/ log3. Then 



y^rlogr > - log (r | X,pD,s) > y ^^^^^^ rlogr 

and 

for s > l,s r > To, where C2 > depends on s and A. We conjecture that the orders on 
the right-hand side, respectively, are the correct ones. 
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The theorems and examples presented in this subsection complement results from [2], 
where general real-valued Levy processes are studied. The main result for compound 
Poisson processes in that paper states that, for any compound Poisson process with 
Elog maxfl 1) < oo and all s > 1, 

logL'^''^(r I X,p-D,s) ^ —r. 

No result on the quantization error for compound Poisson processes is obtained in [2]. 

Our findings also improve the results in [18], where an upper bound for the quantization 
error of real-valued compound Poisson processes is obtained. In particular, it is shown 
that for the Poisson (counting) process and all s > 1, 



-\ogD'-''>{r I X,pd,s) > y-rlogr. 
The correct rate on this scale is given in Example [161 



2 Upper bounds 



In this section, we provide the proofs of the upper bounds for the quantization error and 
the entropy coding error stated in Theorems [TJ [21 and [21 respectively. In the proofs, the 
following four technical lemmas are needed. 

First we prove a result on the asymptotic behaviour of a certain sum occurring in the 
calculations. 

Lemma 19 Let c > 0. Then 

log -— e~''e"''/^''"^^M ~ — \/2rlogr, as r ^ oo. 



Proof: Let V he a random variable that is Poisson distributed with mean c. Then the 
term in question equals 

logEe-"/(^+^). 

By the so-called de Bruijn Tauberian theorem (cf. [1], Theorem 4.12.9), considering the 
Laplace transform is equivalent to considering the lower tail of {V + 1)~^. Thus, consider 

iogpf-^<£) =iogpfF>--i) =iog E 
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where we used Stirling's Formula in the last step. Using the above-mentioned Tauberian 
theorem returns the asserted order of the Laplace transform, including the constant. □ 

Secondly, we prove a quantization result for random variables in a space E with known 
metric entropy. This is needed in order to encode the increments of the process X. 

Lemma 20 Let X be any random variable on a space E. Then, for all s > and all 

e>0, 

D^'^\\ogN{E,p,e) I X,p,s)<e. 
In other words, d'^'^\e \ X,p,s) < N{E,p,e). 

Proof: For given e > let C be an e-net of {E,p). By the definition of the covering 
numbers, C can be chosen to contain only N[E, p, e) elements. Thus 

/ \ V« 

L'(^) (log N{E, p, e) \ X,d,s) < (E min p(X, a)M <e. 

□ 

Remark 21 By using product quantization, it is clear that for a random variable X in 
E^ := E X . . . X E with p^{x^y) := maxi=i^...^fc p(xj, we have 

D^i\k log N{E,p,e) I X,p^s) < e. 



Essentially the same technique is applied in the proof of the next lemma. The result is 
comparable, but slightly more precise. This version is used to encode the jump positions. 

Lemma 22 Let Y be any random variable in [0, l]'^. Then, for all s > , r > 0, 

// Y is such that Yi < . . . <Yk almost surely then we can restrict ourselves to codebooks 
C with Yi< ...<Yk for all Y eC. 

Note that this may be a fairly weak estimate in concrete cases; however, it holds for 
all /c > 1 and all r > 0. If more is known about the distribution of Y , much better 
(asymptotic) estimates are available, cf. e.g. Theorem 6.2. 

Proof: Let us first consider the case e*" = n = ((m + l)/2)'^ with m > 1. Then we can use 
a simple product quantizer. Namely, we set 

C := {(yi/m, . . . , yk/m) E [0, 1]^= : G {1, 3, 5, . . .}, z = 1, . . . , k}. 
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Then #C < ((m + l)/2f and thus 

L>(«)(log((m+l)/2)*^ I y,||.|L,s) < fEmin||r-a|r <m-\ 

For any r > with e'' > 2'', there exists an m > 1 such that (^^)^ < e'' < (^^)'^. Then 

^^'Hr I y,||.|L,«) <^^'niog((m + i)/2)^ I y,||.|L,s) 

where we used > 2^ in the last step. 
Finally, for 1 < e'' < 2^ 

D^'\r I IML > ^) < (E r - (1/2, . . . , 1/2)||:^)^/^ < 1/2 < e-^l\ 

□ 

The last lemma can be strengthened if it is known that the random vector Y = (Fi, . . . , Yk) 
satisfies Yi < . . . <Yk. 

Lemma 23 There are absolute constants c*, /t > such that, for any random variable Y 
in [0, 1]^ such that almost surely Yi < . . . <Yk, we have, for all s > 0, 

^^^H^ I ^> IMIoo'«) < forallr>c*k. (12) 

rv 



Proof: Let m> k and consider 

C ■={(—,■■■ , — ] ■ <yi <y2 < ■ ■ ■ <yk <m,yi e {1, . . . ,m}\ . 

Clearly, 

Note that for any y G [0, 1]^ with |/i < • • • < we have min^gc Wv ~ ct|loo — l/''^- Thus, 

L'(^)(log(#C) |y,||.||^,s)<l, 

777- 

for any random variable Y that satisfies the assumption of the lemma. 
Note that, by Stirling's Formula, for some absolute constants Ci, C2, c* > 0, 

(k+l + k-l\ m\ 2V^^W2^ .k 
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Let r > c*k. Then there is an m > A; such that 



''" + *-^)<e'<('" + \+*-^). (14) 



because, as seen in (US 



''m + k-l\ /k + k-l\ /k+l + k-l" 



Thus, 



k ^ 

By f fT4|) and Stirhng's Formula, for some absolute constants C3, C4 > 
m + 1 + — l\ {m + k)\ 



>^,ll-lloo>^) <-• (15) 
m 



k J m\k\ 

(m + A;)™(m + ^)fce~m-fc^27r(m + A;) 



<cjl + ^] (^.h±l. (16) 



A;'^ V rnk 



Observe that ( 1 + — ) < e and that ^^^^^ = -f + — < 2, for all m and A;. Therefore, the 

\ m J — mk km — ' ' 

term in (fT6|) can be estimated by 



k k 

m J. m 



where k is an absolute constant. This implies ke^/^ < Km or 1/m < Ke~'^/'^/k. We deduce 
from fllSp that for any r > c*k f[T^ holds, as asserted. □ 



Now we can proceed with the proof of our first main result. 

Proof of Theorem [Tl Let Xk be a random variable that has the distribution of X 
conditioned upon the event that Nx = k, i.e. X that has k jumps. Let Y be the vector 
m [0, l]'^ with the jump positions of Xk (in increasing order) and Z be the £'*^"'"^-vector 
containing values of the process X^ between the jumps (in the order corresponding to 
when they occur), i.e. the initial value and the k jump destinations. Note that we can 
reconstruct X^ completely from the vectors Y and Z. Thus, it is sufficient to find good 
codebooks for Z and Y. 

Let (5 > 0. By assumption, there is an Eq = eo{6) g]0, 1[ such that for all < e < Eq, 

logN{E,p,e)<{j + 6)\ogl/e. (17) 
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Let r > logl/eo- For < A; < A;o := koi6,r) := rmin(l, (log l/eo(5))"^) - 1, let C^' be a 
codebook for Z in p'^^^) with 

fEminp'=+^(Z,Z)^V < 2e-"/('=+'). (18) 
V J 

By Remark [211 C^' can be chosen such that 

log#C^'< (A: + l)logiV(E,p,e-"/(^+^)) < (fc + 1)(7 + 5) log (e"/(^+^)) = + 6)r, (19) 
where we used (ITTl) and the choice of fco- 



For 1 < A; < /cn, let be a codebook for Y in 



with 









E min 













By Lemma [22], can be chosen such that log < r — fc. 



(20) 



Define Co := Cq. For 7^ 0, let be the Cartesian product of the codebooks and C^'. 
Then log < r — /c + (7 + 5)r for all < < /cq- 

Let us define the following notation: for any F G C^, we set 



F :-- 



U [^-^^ 



U 



Y Y- 



C[0,1[. 



(21) 



Note that on [0, 1[\F, X can be reconstructed up to the error given in f[T8|) . Furthermore, 
note that the Lebesgue measure of F is less than k Y — Y 

With the help of this information, we can estimate the error of approximating by Ck when 



EminpD(^fc,a)' = Emin ( [\{X k{t) , a{t)) dt 



< C,E min 

aeCk 



... dt 



[0,1[\F 



< C^E min min ( (wk 



Y ~Y 



.. dt 



p^+i(Z,Z) 



: C, {kwY^ min 



Y -Y 



+ Emin p^+^(Z, Z)^ 
zee" 



< (^(2A;«;)" (e"-'^) ''^ + 2^e-''^/('=+^) 



(22) 



(23) 



16 



having used (120]) and (ITS]) in the last but one step, where D := Cs2'*((ew)'* + 1). 
We define the codebook C := Uo<fc<fco ^fc- Then 

oo 

< ^ gr-fc+(7+<5)T- ^ gr+(7+<5)T- ^ ^-fc ^ gr+(7+(5)r+l 
0<fc<A;o fc=0 

Thus, 

D('')((l + 7 + 5)r + l I X,pD,s)^ <EminpD(X,a)^ 

< ^{Nx = k)EmmpniXk,ay+Y.FiNx = k)Emmpj,iXk,ay. (24) 

0<fc<fco k>ko "^'^o 

Using ([2]), (123]) . and the trivial fact that pniXk, a) < w, the last expression is seen to be 
less than 



Ke-^ I re-'' + D J[ k'e-''/^^+^^ + E ^ ^-rs/{k+i)^rs/{k+i) 

l<fc<fco ■ k>ko 

< Ke-^ ire-'' + ^ E 77 e^^e-'-^/^'^+i) + f-V E 77 

V l<fc<fco ^' ^^0^ fc>fco ^' 

oo („s\\k 

< Ke-^Te-'^ + Cx,.,.,A,eoW E e'^^^— (25) 

fc=o 

Recall from Lemma [TO] that the exponential order of the sum, when r oo, is 



— Y2rslog(rs) ~ — y2rslogr 

and that the constant in front of it does not depend on r. The first term in (125]) also has 
no infiuence. Thus, for any 5 > 0, 

logD(«)((l + 7 + 5)r + l I X,pD,s) ^ /2 

limsup -j== < —\ -. 

r^oo vrlogr V s 

Therefore 

logD('^)(r I X,pD,g) ^ / 2 

limsup , , < -\ —r, FT, 

r^oo i/rlogr \s{l + 'y + dj 

which holds for any 6 > 0. Letting 6 tend to gives the assertion. □ 

Proof of Theorem 12 First we treat part (a). Again we condition upon the event that 
k jumps occur. Let Xk be a random variable that has the distribution of X conditioned 
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upon the event that Nx = k, i.e. that X has k jumps. Let, as above, Y be the vector in 
[0, 1]'' with the jump positions of and Z be the ii^'^"'"^-vector containing the values of 
the process X^ between the jumps. Recall that one can reconstruct X^ from Y and Z, so 
it suffices to find good codebooks for Y and Z. 

Let c*, K > be the absolute constants from Lemma [231 Let r > Ac*. Fix k > 1. Let 



be a codebook for Y in (M 


k 1 


■ II oo) 


with 










E min 


Y 


-Y 










oo / 



^) - k k 



By Lemma [251 and the fact that Yi < . . . <Yk,C'j^ can be chosen such that log < kr/X. 
Furthermore, for k >0, let C^' be a codebook for Z in {E^^^^p^^^) with 



E min p*^+^(Z, Z)' ) < 2e-'''^. (26) 



{Z,Zy\ <2e-"/\ 

By Remark [211 C^' can be chosen such that 

log#C^' < (A; + l)logiV(E,p,e-^'/^). (27) 



Let Ck be the Cartesian product of the codebooks C'^ and C[f. Then log#Cfc < A;r/A + (A; + 
l) log A^(E, p, e""^/^). 

Let F be defined as in (1211) . In case k jumps occur {k ^ 0) we approximate X by a 
function from Ck, which gives an error of at most 



EminpD(Xfc,a)' = Emin ( [\{Xk{t), a{t)) dt 



<CsEmm I ( / ... dt 



< C^E min min ( [wk 



... dt 



[0,l[\i^ 



Y -Y 



Cs (kwYE min 



Y -Y 



+ {p'^\Z,Z) 



+ Emin p'=+^(Z, Zf 
z&C' 



{2i 



(29) 



For = 0, set Co := Cq. Then the error is less than 2e ^^Z^, by ( l26l) . 
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On the other hand, this procedure has an expected nat length of at most 
Ke-^ E 7T {i^Cu) = KY^-e-' (kr/X +{k + l) log iV(E, p, e"^/^; 

fc=0 k=0 

= K(r + {\ + l) log N{E, p, e-'l^ 

Therefore, similarly to 



D('^)(ir(r + (A + l)logiV(E,p,e-'-/^)) | X,p^,s) 



(oo \k \ 

Coo x A; \ 
2e— /Ag-A ^ ^^2"((y,^)- + l)e-''^/^ — e"^j < + l)e-''''^. (30) 

where only depends on s. This yields the assertion (a). 

To see (b) one only has to recall that in case the jump positions are distributed as a 
Poisson point process we can choose K = 1 m 

Let us finally show (c). In the case of a discrete space E = {xi, . . . ,Xq} with w = 
maxa;^yg£; p(x, y), we can choose log #C^' = (A; + l)logg. Thus, on [0,1 no error arises. 
This allows to replace the right-hand side in (129!) by {2wK)^e~^^^'^. Therefore, the upper 
bound in (!30l) becomes K(2wk)'^{1 — e~^)e~^^/^ ^ where k > is the absolute constant 
from Lemma [231 This finishes the proof of (c). □ 



Note that no assumption is necessary on the correlation of the jump positions and incre- 
ments. 

Let us now indicate the changes that are necessary to prove Theorem [31 

Proof of Theorem [31 The proof carries over almost literally from Theorems [H and [21 
respectively. The only differences concern the assumption on S-'^^ instead of the metric 
entropy iV, the fixed initial position, and the possibly unbounded jumps. 

In this case, we encode the increments instead of the jump destinations. Let Y be as 
above, but Z denote the vector with the increments, i.e. Z := (Z^^^ . . . , Z^'^)) with 
:= X{Y,;)-X{Yi-). Note that we can reconstruct X from Y and Z, since we asssumed 
X(Q) to be deterministic. 

The first change is to replace (IT^ by 

log#C^' < A;logrf(^)(e-"/'= | Z^^\p, s) < k{-f + S) loge"/*^ = (7 + S)r 

in the proof for the quantization error. For the entropy coding error one has to replace 
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(EZDby 



log#4'<fclogci(«)(e-'/^ I Z«,p,s). 



The second issue concerns a certain refinement in order to deal with the possibly un- 
bounded jumps. Here, we need that we deal with a normed space. We will show that, on 
average, the high jumps do not have any influence on the rate. In fact, the only modifi- 
cation affects fl22l) . where we estimate by 



CsE min min 



max 

l<n.m<fc 



i=l 



i=l 



k 



Y -Y 



+ {kp'{Zj) 



which is required due to the fact that we cannot estimate by a finite diameter w (modi- 
fication in the first term) and the errors may add up over all the jumps, since we encode 
the increments and not the absolute positions (modification in the second term). 

The first term can be estimated by 

k k 



/ k k 

Cs^ min min V 11 - Z^'^ I + V \z^^ 
Yec'^ z^c'l Vi=i " ' i=i " 



<Cl FEminpMZ,Z + A; 



Y -Y 





^(1) 


^ I /c'E min 


Y -Y 











< Cl{2' + E Z(^) ')A;2^+^E min 



Y -Y 



where the last step comes from (ITS]) . 

This leads to an additional factor Ck'^^^ in fl25]) which has no influence on the order. 
Note furthermore that this argument needs that the jump positions and the increments 
are independent (in order to separate the expectations) and that the increments are 
identically distributed (as Z^^"*). It is not needed that the increments are independent 
among each other. 



Analogously, for the proof of the entropy coding error, fl28|) is modified, which leads to an 
additional factor of Ck^^^ in fl2^ . which leaves the resulting order unchanged, but which 
does change the constant. □ 



3 Lower bound for the quantization error 



In this section, we prove the lower bounds for the quantization error. Essentially we employ 
a small ball argument, i.e. we construct an event of not too small probability that still 
leaves sufficient uncertainty for the error to be large. 
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First we prove Theorem |H 



Proof of Theorem |4l Let us fix > and S > (to be chosen later) and define intervals 
Ij := + ^, i - ^1 , j = 1, . . . , A;. Note that Xi{Ij) = l/{2k). Let A be the event that 
X has exactly k jumps at Yi, . . . , Y^, such that Yj G Ij, for all j = 1, . . . ,k, and that the 
moduli of the increments are all greater than Eq- Since the Yi and Zi are independent (by 
condition (*)) and the Yi are distributed according to a Poisson point process, we have 




Step 1: Let Xa be a random variable with the distribution of X conditioned upon the 
event A. Then 



D(^)(r I XA,PT,,sy= inf Ex,minpD(X^,/)^ 

log{#C)<r /GC 

> inf 5^P(V/gC : p^{XaJ)>5) 

log(#C)<r 

= inf 5^ (1 - P (3/ G C : Pd(X^, /) < 5)) 

log{#C)<r 

> inf 5^ f 1 - (#C) supP (pd(X^, /) < 5)] 

>5'[l- e'- sup P (pd(X^, /) < 5) j , (31) 
where the supremum is taken over all functions / in D([0, 1[, E\ For such /, we have 
P (pd(Xa, /) < 5) = P p{XA{t), fit)) dt < 6^ 

< P f n piXAit), fit)) dt < (jj j = EP f n 1 1 piXAit), fit)) dt < S 



Z , (32) 



where Z = (X^(0), Xa(Yi), . . . , Xa(Ya;)) is the vector with the jump destinations. By 
condition (*), we have that, conditioned upon Z, the events 

p(X^(t),/(t))dt<5|y 

are independent, since each of them only depends on the jump position in the respective 
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interval. This together with (15^ shows 



k / 

supP(pD(X^,/)<5)<supEnP / p{XA{t)J{t))dt<6 

f f j=i 



Z . 



(33) 



Step 2: Now we estimate each term in the product separately. Fix j G {1, . . . ,k}. Define 
Ij := + j^, i.e. the left end point of the interval Ij. Furthermore, we define 

Bj := {t G /, : p(X^(/,), f{t)) < eo/2} and C, := {t G I, : X^(t) = X^(/,)} . 

Then we show that 



p{XA{t),f{t))dt<6 



^0 



(34) 



where BjACj := (5| n Cj) U {Bj f] C|). Indeed, assume that we had Xi{BjACj) > 2S/eo. 
Then 



p{XA{t),f{t))dt 



> 



[ p{XAit), fit)) dt+ [ p{XA{t), fit)) dt>^ Xi{B,AC,) > 5, 



where we used the triangle inequality in the last but one step. This shows and we 
thus have 



P 



piXAit)J{t))dt<5 



Zj <PMAi(i?,)-Ai(Q)|<^ 



Z . 



Note that, conditioned upon Z, \i{Bj) is a deterministic value (depending on X^(Zj) and 
/), whereas Xi{Cj) is a random variable that is uniformly distributed in [0, l/{2k)], since 
the point in Ij where the jump of Xa occurs is uniformly distributed in Ij. Therefore, 



p(|Ai(i?,)-Ai(Q)|<- 



Z < 



86k 

^0 ' 



Step 3: This shows, continuing ([33]), that supy P (pd(^a, /) < S) < {8k5/£o)^. Substitut- 
ing this estimate back into fl3T]) . we obtain 



D(^)(r I Xa. pd, sf > 5' [l - e'{86k/eo) 



Therefore, 



D^'^\r I X,pD,s)^ > P(A) ■ D^'^\r \ X^,Pd,s)^ > (^^^ 6"^ (l 



' 8k6 ' 
. ^0 . 
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Now we can optimize k > 1 and 5 > to obtain the largest possible lower bound. We set 



8k \2 

Then the last estimate becomes 



We set 

k := [\j2sr/ log rj ~ \j2sr / logr. 
Taking logarithms of the last estimate shows that 



k 2 

— logD^^Ur I X, pd, s) < — log/c + r/k W - r logr, 

s \ s 

as asserted. □ 

The proof of Theorem contains the same idea as the one of Theorem [Hand carries over 
almost literally. Therefore, we only indicate the necessary changes. 

Proof of Theorem I14t By assumption, Z^^^ has an absolutely continuous component. 
Let S" C M'^ be a measurable set with Xd{S) > on which Z^^^ has a positive bounded 
density w.r.t. the Lebesgue measure and such that ^ S. Define Eq := dist(S', 0)/2 > 0. 

This time, A is defined as follows: let A be the event that X has exactly k jumps at 
Yi, . . . , Yk, such that Yj G Ij, for all j = 1, . . . , k, and that the corresponding increments 
(i.e. Z^^^ = X{Yj) — X{Yj—)) are of a height in S. Due to the Poissonian nature of the 
point process and since increments and positions are independent, we have 



k 

P (A) = [] P (exactly one jump in Ij) ■ P {X{Yj) - X{Yj-) E S) 
■P(no jump in [{j - l)/k,j/k] \ Ij) 



where qs := P [Z^^^^ G > 0. Regarding fl3Tl) . the proof is analogous to that of TheoremHl 
We set Z = (X^(Yi), . . . , X^(Yfc)) for the vector with the jump destinations. In 
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and (1551) we estimate a bit more carefully and obtain: 



P(Pd(Xa,/) <5) 



< E n p / pix^if), fit)) dt < s, r p(x^(t), fit)) dt < s 



z\. 



As in the proof of Theorem HJ the sets Bj and Cj are introduced and (IMl) is established. 
Let r^- : = i /k and := r^- — 1/(4/;;). Because of ( 134|) and since = X^(r^.) = XA(rj) 

on [rj,r'], the last expression is less than 



E n P MBACj) <5, r piXAirj), fit)) dt < 5 



Note that, conditioned upon Z, the events XdiBjACj) < 6 and /r/ p(Xyi(r_,), fit)) dt < 6 
are independent, since the second event only depends on Z, i.e. it is deterministic. Thus 
the last expression equals 



k /I 

EnP(A,(5AC) <5|Z) P r>(X^(r,),/(t))dt<(5 



The first term can be estimated as in the proof of Theorem HI by S^Aj/eo? which allows to 
estimate the last expression by 



k 



EllP r piXAir,)Jit))dt<6 
j=i V^j 



In order to treat the second term, note that it equals 



P( / piXAir,)Jit))dt<6,j = l,...,kj 

= EP p(X^(r,), fit)) dt < 5, J = 1, . . . , 



Z^'\...,Z^'-'n. (35) 



Note that the last condition (for j = k) is the only non-deterministic condition in the 
probability. It depends on Z^''\ which is an M'^- valued random variable distributed as Z^^\ 
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By the definition of tlie event A, Z^^^ attains values in S. Tlius, 



P 



p(X^(r,),/(t))dt<5 
= P 



z« z^'^-i) 



(r 


k 


dt<5 








oo 





< p 



r' k~l 



Ak 



•^''fe 7 = 1 



< 5 






oo 







where the integral is to be understood componentwise. Note that I]y=i ^'^''^ ~ /(^) dt 



is a deterministic value in M'^, conditioned upon {Z'^^\ . . . , ^''). Thus, the last term is 
bounded from above by £[,(8^5)'^, where e'^ is the supremum of the density of Z^^^ = Z^^^ 
in S. In the same way, successively the other terms can be reduced; and the expression in 
fl35l) can be estimated by e'Q{^k5Y^. Therefore, 

f 

where Eq = 8 min(l/£o, £^o)- Continuing as in Step 3 of the proof of Theorem H] shows 
D^'^\r I X,p-D,sr > P(A) • D^'^\r \ Xa^Pd^sY > (^^^ (l - (Me 



o) 



This time we set 



Then again 



1 n ^i/(fe(i+rf)) 



I. Y.po..)>(t) 



k/s 



where this time we set 



k :-- 



2s r 



1 + d log r 



2s r 



1 + d log r 



This eventually leads to 



-logD(9)(r I X,pD,s) <-\ogk + r/ikil + d)) ~ /_-l_rlogr, 

s V s(l + d) 



as asserted. 



□ 



25 



4 Lower bound for the entropy coding error 



In this section we prove a corresponding lower bound for the entropy coding error (in fact, 
for the distortion rate function) for a jump process where the underlying point process is 
Poissonian. We use the notation from [T^, in particular, for the distortion rate function 

D{r I X,pn,s) := inf |(Epd(X, X)^)'^' : /(X;X) <r|. 



and the notion of mutual information: 
/(X;X) = 




^dP;,^^ if P;,_^ <Px®Px 
otherwise. 



We recall that D{r \ X, p, s) < D^^\r \ X, p, s) < D^'^\r \ X, p, s) for any random variable, 
all moments and any distortion measure. Therefore, a lower bound for D immediately 
translates into a lower bound for D^'^^ 

Let us state the assumptions of the main result of this section. We shall require that X is 
a jump process (on the index set [0, 1[) whose jumps form a Poisson process of intensity 
A > 0. Furthermore, we assume that p defines a metric on E and that the moduli of the 
jumps of X s. bounded from below by a constant Eq > 0. 

As before, we denote by (Yi) the jump times of the process X and by Nx the random 
number of jumps of X; we set Yq = 0. Moreover, we assume that conditioned upon 
Nx = k the random vector (Aj) := (X(Fj))*LQ and the jump times (1^) are independent 
(condition (*)). In the rest of this section, we prove the following stronger version of 
Theorem [3 

Theorem 24 Under the above assumptions one has 

D{r I X, pD, 1) > e^C min(l. A) e""'^, 
where C > is an absolute constant. 

Let us shortly describe the idea of the proof. We relate the coding complexity of the jump 
process to that of the random jump times. Controlling the complexity of the jump times 
by using Shannon's lower bound then leads to a lower bound in terms of a variational 
problem. The proof is based on several lemmas and a particular random partition of [0, 1[. 

We denote by the dyadic subintervals of [0, 1[ of the m-th level, that is 

P„ = {[(j-1)2— ,j2— ):j = l,...,2-} 

We construct for any collection ti, . . . , of distinct points in [0, 1[ a finite binary tree as 
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follows. Let /i = Sti^ with Sz denoting the Dirac mass in z. The root of the tree will 
be associated with the interval [0, 1[ and it will be marked by the number /i([0, 1[) = k. 
If A; G {0, 1}, then the construction ends and the root is also a leaf of the tree. If > 2, 
the root of the tree is attached two children namely the two dyadic intervals of T>i that 
are contained in [0, 1[: [0, l/2[ and [1/2, 1[. Again we mark each of the nodes with their 
corresponding masses. Each node that has mass or 1 becomes a leaf of the tree, and for 
each node with mass greater than 1 we attach the two dyadic intervals of the next level 
that are contained in the interval and we continue in analogy to above. 

By the construction, each leaf contains either one or no point. We shall denote by 
7rfc(ti, . . . ,tk) := (/i, . . . , Ik) the fc-intervals associated to the leaves with positive mass. In 
order to make the definition unique we arrange the intervals in their natural order. 

Lemma 25 Let k > 1 and (/i,...,/^) G im(7r;;). Conditioned upon the event {Nx = 

k, 7rfc(Yi, . . . , Yfc) = (Ji, . . . , 4)} we have that (Yi, . . . , Ifc) = {Ui, ...,Uk), where Ui are 
independent random variables that are uniformly distributed on li, respectively. 

Proof: First note that for any collection of distinct points ti, . . . ,tk G [0, 1[ such that 
Yl^=i ^liitj) = 1 for alH = 1, . . . , one retrieves Tik{ti, ■ ■ ■ ,tk) = (Ji, . . . , Jfc). On the other 
hand, any collection of points which yields Z]j=i ^hi'tj) 1 one i, induces a different 
tree and 7ik{ti, . . . ,tk) ^ {h, . . . , h)- 

Therefore, the following two events coincide 

k 

{Nx = k,nkiY^,...,Yk) = ih,...,Ik)} = {Nx = k,Y,hXY,) = lfoTZ = l,...,k]. 

Recall that the times {Yi) form a Poisson process on [0, 1[ so that conditioned on {Nx = 
k, 7r(Yi, . . . , Ffe) = (Ji, . . . , 4)} one has (Yi, . . . ,Yk) = {Ui, . . . ,Uk), where Ui are indepen- 
dent random variables uniformly distributed on Jj. □ 

Lemma 26 Fix k > 1, (/i, . . . , 4) G im(7rfc) and distinct points Oq, . . . , ctfc ^ E with 
\ai — flj-il > Eq for i = l,...,k. Moreover, let /i denote the distribution of a process 
in D{[0,1[, E) that has jump positions at k uniformly distributed times in the intervals 
4, • • • , 4 one? that attains the values Oq, . . . , in the given order. Then 




Proof: With slight abuse of notation we shall denote by X = (X(t))tg[o,i[ a yU-distributed 
process and we let Fi, . . . , Yk denote the ordered k jump positions of X. Due to Lemma 
[25] the times Yi, . . . , are independent and each Yi is uniformly distributed on /«. 
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Now let X = {X(t))t£[o,i[ denote a D([0, 1[, ii^)-valued reconstruction with I{X;X) < r. 
We define Xi = a^-i for t < Yi and Xi = for t >Yi. Also we set Xi = X{t) for t e li 
and XI — XI otherwise. Then clearly 

k 

Pd(X,X)>^Pd(X\XO. 

i=l 

Next, we will provide a lower bound for the right hand side in the latter inequality. 

For each fixed i = 1, . . . , A; we define Vi to be the probability kernel of the regular con- 
ditional probability 'S'(Yi G ■\X = ■). Next we choose Yi = Yi{X) to be the first time 
t e [0, 1[ for which the probability i'i{X, [0,t]) is greater or equal to the threshold 1/2. 

We observe that for t E [0, 1[ 

E[p{Xl, XI) \X] > P (xi = a,_i|x) A P (xi = a,|x) [p(a,_i, XI) + p{ai, XI) 
> P (xi = ai_i\x) A P (xi = ai\x) p{ai_i, a^) 

= £o P (>i < ^1^) A P (Fi > . 

Consequently, the approximation error satisfies 

E[pu{X\X')\X] > eoj\(Yi < t\x) A P (f^ > t\x) dt 



E[%<i}|X]dt+ / E[%>i}|X]dt 



Yi 



eoE{\Yi-Yi\ \X) 



and one gets 



E[pj,{X,X)]>eo^E\Y,-Yi\ 



(36) 



We shall now use the Shannon lower bound to derive a lower bound for the right hand 
side of the latter equation. For ease of notation we write shortly Y — {Yi, . . . , Yk) and 
Y = (Yi, . . . ,Yk). We need the notation for the continuous entropy and its conditional 
counterpart: for M'^- valued random vectors Z and Z we denote 

r HP r dP 

h{Z) ■.= -J log^dPz and h{Z\Z) := - J log^^dP^_^, 

provided the Radon-Nikodym derivatives exist and the integrals are well-defined. 
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Since Y is cr(X)-measurable we have I{Y;Y) < I{X;X) < r; so that by the Shannon 
lower bound 

r > I{Y;Y) = h{Y) - h{Y\Y) 
= h{Y) - h{Y - Y\Y) > h{Y) - h{Y - Y). 

In particular, Y — Y is absolutely continuous and its differential entropy is well-defined. 
Next, we set d := IE||y — Y\\^k and estimate the term h{Y — Y) from above by 

(f){d) = sup h{Z). 

Z, Vz < Afc 

1 

Using Lemma 6.4 from [T] (which is based on ideas from jS]) one can easily show that 

k 



*LjE|Zi|<d i=l \ / 



sup 

Z, Pz « 

Consequently, r > h{Y) — k\og{2ed/k) or, equivalently, 

- 2e 

Moreover, the entropy of Y satisfies 

h{Y)=j:h{Y^= j:\og 

and we conclude that 

d > , 
2e 



i=l 

which together with ( |36l) shows the assertion. □ 

A crucial quantity in the latter lower bound for the distortion rate function is the length 
of the intervals /». Later we will use the following estimate: 

Lemma 27 Let ti, . . . ,tk G [0, 1[ denote k distinct points ordered by their size and let 
(Ji, . . . ,Ik) = iTkiti, . . . ,tk). With to = —oo and tk+i = oo we get for each i = 1, . . . ,k 
that ^ 

m > -{ti-ti_i) A (ti+i-ti). 



Proof: By definition Jj is the largest dyadic interval that only contains the point ti and 
the assertion follows since all half-open intervals of length {ti — ti-i) A (tj+i — ti) that 
contain ti do not contain any of the other points. □ 
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Lemma 28 There exists a universal constant cti G M and a function 02 : N — > M such 
that for any k > 1 and i E {1, . . . , k} 

E[log[(r, - A - Yi)]\Nx = k]=ai- a2{k). 

Proof: Let Yi, . . . ,Yk denote the order statistics of k independent [0, 1 [-uniformly dis- 
tributed random variables, and let (Fj) denote the random jump positions of a Poisson 
process of intensity 1 on [0, oo[. First let z G {1, . . . , — 1} 

E[log(F, - A - Y^\Nx = k] = Elog((f, - Y,^^) A {Y,+, - Y^)) 

= Elog(^^A%^) 

^ J-k+l J-k+1 ' 

= Elog((Fi - Fo) A (F2 - Fi)) - E[logn+i]. 



For the second equality see e.g. [5], Proposition 13.15. 

Setting a\ := Elog((Yi — Yq) A (12 — F)) and a^ik^ '■= E[log F^+i] finishes the proof in 
this case. The statement follows analogously for i = k. □ 

Furthermore, we will need asymptotic estimates for 

A:=Nx( - Y^-l) A (F+1 - Yi] 

and 

Nx Nx 

Rp := Nx\og^(3{\{{Yi - A (F+i - Y,))^""^ = \og^^(3{Y, - F-i) A (F+i - F), 

i=l i=l 

where > 0. 
Lemma 29 One has 

ER^ > X\ogf3 + c 

for the constant c = c(A) = Xai — E.[Nxa2{Nx)] G M, where ai and a2 are as in the 
previous lemma. Moreover, 

lim (3E\^Aa] = a. 



Proof: Applying Lemma [281 we get 



Nx r 



ERp = E[E Rp\Nx]] >EJ2 



i=l 



log/3 + E log(F - F_i) A (F+i - Yi)\N^ 



X 



EN 



X 



log/3 + ai- a2iNx) 



X\og/3 + c. 
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The second statement is an immediate consequence of the monotone convergence theorem: 
since A > a.s. one has 



/3E 



X 



/3 



A A 



E[Nx A PA] ENx = A. 



□ 



We are now in the position to prove Theorem 

Proof of Theorem I24t Let X be D([0, 1[, -E')-valued reconstruction with /(X; X) < r 
for some fixed r > 0. Furthermore, we denote by 

G{k, (Ji, . . . ,4), (ao, . . . 

= J(X; X\Nx = k, nk(Y) = (Ji, . . . , 4), {Aq, ...,Ak) = (ao, . . . , a^)) 

the conditional mutual information of X and X given Nx, ttnx(X)j ^'^^ (^O; • • • ? ^a^x)- 
We consider the non-negative random variable R = G{Nx, nN^(Y), {Aq, . . . , A^)). 

Since {Nx, TCNxi^)} (^o, • • • , Ak)) is cr(X)-measurable one has 

r > I{X; X) > I{X- X\Nx, vr^x(l^), (^o, ...,Au))=ER. 

Moreover, Lemma [26] together with Lemma [27] implies that 



Epd(X,X) > 

4e 



Nx 



Nx[\{{Y,-Y,_{)A{Y,+^-Yi)] e 



i=l 



In order to get a lower bound for the coding error we next analyze the minimization 
problem 



E 



'Nx 



l/Nx 



Nx\\{{Y.i-Y,_{)A{Yi^^-Y,) 



-R/Nji 



mm! 



where the infimum is taken over all non- negative random variables R satisfying KR < r. 

We let again A = Nx{Ili^=i{Yi — Yi-i) A (Vj+i — Yi)y^^^ . Using Lagrange multipliers one 
gets that for every P > 

BA /'^^ 
R0 = Nx log+ ^ = Nx log+ /5 n(^^ - ^-i) ^ (^^+1 - ^0)'/^^ 

is a minimizer when r = r^g := Ei?^ = E[A^x log^ j^]. Moreover, elementary computations 
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give that the corresponding minimal value in the minimization problem is 



dp := E 



A exp — log 



/3A' 



E 



N- 



X 



X 



/3 



A A 



For given r > we now choose (3 = (3{r) = exp((r — c)/A) where c is as in Lemma 
Then r = A log P + c < ERp and due to the variational formula above one has 

fl(.|A-,po.l)>|E(|;^AA). 

Thus letting r tend to infinity we get 

^(r|X,p,,l)>|^ = |Aexp(-(r-c)/A). 

Thus, one has for all sufficiently large r that 

D{r I X,pD,l) > ^Ae-<=/Voe-^/" = Cxe,e-'-'\ 
8e 

where 

Cx = ^ e''^ and c = Aai - E[Nx a2{Nx)]. 

Moreover, ai and 0:2 can be expressed in terms of i.i.d. standard exponential random 
variables (cj) as ai = Elog(ei Ae2) and a2{n) = ElogYl^=i Gi, cf- the proof of Lemma [281 

After some calculations (using Mathematica) one obtains 

T'(n + 1) 



roc 

ai = (logx)2e~^^ da; = —7 — log 2, a2(n 
Jo 



T{n + 1) 



where 7 = 0.57721 ... is the Euler-Mascheroni constant and F is the Gamma function. 
Some more calculations show that 



c/A = — 7 — log2 — log A — y x~^e~^dx — 



1-e 



-A 



Closer analysis of this term shows that 



\imC, = ^e-\ limC,/A=-^ 
A-*oo 8e A^o 8e 



which altogether shows that C\ can be estimated from below by Dmm{l,X) with some 
absolute constant D > 0. □ 
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